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How Pilot Schools Authentically 
Assess Student Mastery 

EXECUTIVE SUMMARY 

This study documents how member schools of the Boston Pilot Schools Network use 
authentic assessments to understand what their students know and can do. The Boston 
Pilot Schools were created in 1994 in response to charter school legislation to increase 
choice options within the Boston Public Schools (BPS). The Network currently 
encompasses nineteen schools and almost 10% of the district’s total 61,000 students in 
130 schools. Most Pilot Schools are small, and all are vision driven, and autonomous. 
Compared with other district schools, Pilot Schools have among the highest attendance 
rates, longest wait lists, lowest numbers of transfers, and lowest suspension rates in BPS, 
all indicators of high student engagement (CCE, 2001). On performance indicators, Pilot 
Schools reported low grade retention rates and higher rates of graduation and college- 
going as compared to BPS (CCE, 2001), and rank among the top Boston schools on 
standardized tests. 

Staff in Pilot Schools believe that understanding what their students know and can do is 
best achieved through authentic assessments. Authentic assessments involve the making 
of meaning, guided inquiry, and value beyond school (Newmann, 1995). Against a 
backdrop of state-mandated standardized tests at more grade levels and in more subjects, 
and federal legislation in the form of No Child Left Behind, Pilot Schools use 
performance-based tasks in which students ask questions that they have formulated on 
their own and use habits of mind to reflect on their work and thinking. 

Our research on Pilot Schools revealed three major modes of authentic assessments— 
evaluations, presentations and celebrations — each used for different contexts and 
purposes. We found that: 

• Pilot Schools rely on authentic assessment to make serious decisions about 
student progress. 

• Pilot Schools are at different stages in their development of authentic assessment 
systems. 

• Authentic assessment can improve professional development and increase 
collegiality. 

• Authentic assessment engages the community in the school. 

Such complex and comprehensive systems of assessment inevitably encounter challenges 
in implementation. The rationale for sharing common challenges in the Pilot Schools’ 
implementation of authentic assessment is the same as for sharing the successes — 1) to 
assist the Network schools in reflecting about their practices; 2) to provide information 
which can be useful to practitioners, within the Network, within the district, and beyond. 
These challenges include: 
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• Assessment of content knowledge needs to be improved. 

• Concerns about reliability and standardization of scoring need to be addressed. 

• Authentic assessments must balance the use of students’ strengths and 
weaknesses. 

• Not all performance-based assessments in Pilot Schools are authentic 
assessments. 

• Outside evaluators need more support and training. 

When a network of schools commits to using multiple means of assessing their students’ 
knowledge and skills, both students and educators benefit. While difficult to develop and 
implement, especially with the current reliance on standardized test scores for district, 
school, teacher, and student evaluation, authentic assessments have positive implications 
for all stakeholders: 

• Authentic assessments raise the stakes of learning for students. Students in Pilot 
Schools have opportunities to present publicly to adults and students, both familiar 
and not, throughout their careers. They complete work that is relevant to their daily 
lives and contexts. They can demonstrate what they know and can do in a variety of 
ways, rather than just through a paper and pencil test. 

• Authentic assessments improve curriculum and instruction. Teachers have 
opportunities to refine their curriculum and assessments collaboratively, often across 
subjects and grade levels. They make their practice more open by sharing their 
assignments and assessments publicly. 

• Authentic assessments make public the work of schools in accountable ways. 
Community members, including parents, business, and community leaders, become 
more educated about public schools, curriculum, and assessment. Their participation 
lends credibility to the authentic assessments and add a sense of seriousness and 
purpose. 



Research and Evaluation Program 



ii 



Center for Collaborative Education 




How Pilot Schools Authentically 
Assess Student Mastery 



RATIONALE 

The purpose of this study is to document how member schools of the Boston Pilot 
Schools Network use authentic assessments to understand what their students know and 
can do. Two of the Network’s guiding principles directly address what assessments of 
student learning should look like: 

• Learning should be purposeful, authentic, challenging, and creative, and build 
students' capacity to take responsibility for their own learning; 

• Authentic forms of assessment, such as portfolios and exhibitions, are key to 
improving learning and teaching. 

These principles not only reflect the belief that a student’s learning must be measured in 
multiple, authentic ways, they also reflect the Network’s focus on equitable practices — 
schools should take responsibility to encourage students from different contexts and 
cultures to show in diverse ways what they know and how they learn. 

Our study provides documentation of a new generation of schools that have come of age 
after most of the foundational research on authentic assessment was produced. We offer a 
snapshot of the use of authentic assessment in the Pilot Schools and identify the 
challenges the schools face as they try to advance this work. This study contributes to the 
field by documenting perfonnance assessment schools as they struggle to continue the 
work in a difficult political context. We hope to inform those working with authentic 
assessment or evaluating its impact. Ideally, our work will also be of use to practitioners 
as they implement personalized assessments in an increasingly standardized environment. 

THE BOSTON PILOT SCHOOLS NETWORK 

The Boston Pilot Schools were created in 1994 in response to charter school legislation to 
increase choice options within the Boston Public Schools (BPS). The Network currently 
encompasses nineteen schools 1 and 6100 students in the district, which serves a total of 
about 61,000 students in 130 schools. Most Pilot Schools are small, and all are vision 
driven, and autonomous. Through an agreement with the Boston Teachers’ Union, the 
Boston School Committee, and BPS, the Pilot Schools have unusual, charter-like freedom 
(autonomy) over crucial aspects of their school design and implementation — budget, 
staffing, scheduling, governance, and curriculum. In exchange for these freedoms, they 
have increased accountability for serving their students through a rigorous School Quality 
Review process in which each school’s Pilot status is renewed by the Boston School 
Committee every four years. The Center for Collaborative Education (CCE), a nonprofit 



1 See http://www.ccebos.org/pilotschools/schools.html for descriptions of Pilot Schools 
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organization dedicated to working with networks of urban public schools engaged in 
reform, supports the Pilot Schools by providing them with significant school-based 
coaching and professional development, networking opportunities, advocacy with the 
district, and research and evaluation studies. 

Pilot School students are representative of the district by gender, race, income, and 
mainstream special education status, although they serve a lower percentage of English 
language learners than the district average (CCE, 2001). At the high school level, Pilot 
Schools enroll a higher percentage of African American students than the district 
average. Compared with other district schools, Pilot Schools have among the highest 
attendance rates, longest wait lists, lowest numbers of transfers, and lowest suspension 
rates in BPS, all indicators of high student engagement (CCE, 2001). On perfonnance 
indicators, Pilot Schools reported low grade retention rates and higher rates of graduation 
and college-going as compared to BPS (CCE, 2001). Pilot Schools also rank among the 
top Boston schools on standardized tests. These findings corroborate the research that 
says low-income students and students of color achieve at higher levels in small, 
personalized schools (Raywid, 1998; Cotton, 2001). 

Acknowledging Pilots’ success with students, CCE seeks to understand more fully the 
pragmatic aspects of Pilot School autonomy over curriculum and assessment. 
Specifically, in this study, researchers studied the development of authentic assessments 
to measure the skills and competencies of Pilot School students. 

THE CONTEXT OF AUTHENTIC ASSESSMENTS IN THE 
UNITED STATES 

Definitions of Authentic Assessment 

A shared definition of ‘Authentic Assessment’ has proven elusive. Initially, criticism of 
traditional testing formed the basis of a quest for better assessments, with authentic 
assessment being defined mostly by what it was not. However, a literature review reveals 
some dominant themes that describe authentic assessment. In Beyond Standardized 
Testing (1988), Archbald and Newmann wrote: 

Most traditional assessment indicators communicate very little about the quality and 
substance of students’ specific accomplishments. The type of learning actually measured 
is often considered trivial, meaningless, and contrived by students and adult authorities. 

A valid assessment system provides information about the particular tasks on which 
students succeed or fail, but more important, it also presents tasks that are worthwhile, 
significant, and meaningful — in short, authentic. 

Grant Wiggins, the most prolific advocate of new kinds of assessments, wrote that 
“assessment is authentic when we directly examine student performance on worthy 
intellectual tasks” (Wiggins, 1990). Such tasks, he argues, should require students to 
construct complex responses to assignments that mirror real-world contexts and 
challenges, rather than merely asking students to choose an answer as in traditional 
exams. These assessments should be deliberately open-ended and student work should be 
judged publicly and held to clear and rigorous performance standards. Most importantly, 
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assessments should engender thoughtful reflection by students about their work, as well 
as by the faculty, upon the learning process. This reflection should demonstrate a degree 
of awareness and meta-cognition not valued or instilled by traditional assessments 
(Wiggins, 1989, 1993). 

Newmann (1995) has developed a definition of authentic assessment with three elements: 
construction of knowledge, the use of disciplined inquiry, and value beyond school. 
Construction of knowledge means that students write, build, or perform rather than 
simply identify and reproduce knowledge. Disciplined inquiry means that from a base of 
knowledge, students endeavor to understand a problem deeply and communicate that new 
knowledge. Value beyond school means that the product of authentic work has the 
potential to impact others in some way. Work that is authentic by these criteria improves 
student engagement and problem solving skills beyond the classroom. 

The definitions of ‘authentic assessment’ by other authors tend to reflect these core 
themes. While terms such as ‘performance assessment’ and ‘alternative assessment’ are 
often used synonymously (Darling-Hammond, Ancess & Falk, 1995; Baron & Foschee, 
1995; Herman, Aschbacher & Winters, 1992), not all perfonnance assessments are 
authentic, as in the case of students being asked to produce writing on demand in a 
contrived context (Meyer, 1993). Authenticity, according to most definitions, is best 
achieved when the context is germane to real life in all its complexity, ruling out 
performance-based tasks in more controlled environments. Interestingly, the Office of 
Technology Assessment of the U.S. Congress has defined perfonnance assessment as 
“testing that requires a student to create an answer or a product that demonstrates his or 
her knowledge or skills,” reflecting the extent to which new assessment ideas have passed 
into public discourse at the highest levels (Rudner & Boston, 1994). However, 
proponents of authentic assessment might criticize this definition as enabling better 
standardized testing, not better assessment systems. 

The ideas of these authors have been influential in developing CCE’s working definition 
of authentic assessment, which guides this study. Adopted in 2002, it represents a 
synthesis of major ideas as well as a platform for research. It is presented here in full: 

Authentic assessment allows students to demonstrate their competence in multiple ways. 

Students work on performance -based tasks, using, applying, and expressing habits of 
mind and the knowledge that they have created. Throughout, students are asked to reflect 
on their work and thinking in order to develop their capacity to assess and redirect their 
learning. 

Assessment is authentic when it includes the making of meaning. This occurs through a 
process of inquiry that is informed by prior knowledge and making connections to new 
understanding. Students demonstrate meaning making in original ways that have value 
beyond school, and include the possibility of contributions to the field of study 
(Newmann 1995). Authentic assessments are embedded in ongoing curriculum work, as 
well as in culminating performances. Typical authentic assessments include projects, 
exhibitions, portfolios, and demonstrations (CCE, 2002). 
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History of Authentic Assessments in US Education 

Authentic assessment started out as the localized work of a few schools, was enhanced by 
the theories of academics, and slowly became popularized by researchers. In the 1970’s, 
Walden III, a public alternative school in Racine, Wisconsin, developed a widely admired 
portfolio assessment system (Meier, 1995). Drawing upon the work of John Dewey, the 
school sought to foster student learning by experiencing and doing. At the same time, the 
Prospect School in Vermont, led by Pat Carini, was developing ways of looking at 
student work over time and using it as a glimpse into the learning process (Engel, 1980). 
Two major books furthered thinking about authentic assessment. Howard Gardner’s 
Frames of Mind: The Theory of Multiple Intelligences (1983) suggested that varied kinds 
of assessments would better serve children who learned in different ways; Horace ’s 
Compromise by Ted Sizer (1984) advanced the idea that students should study a small 
group of topics in depth, and that they should be required to exhibit mastery of skills and 
content in order to graduate. Other schools began to take notice, one of which was 
Central Park East Secondary School (CPESS) in New York City, led by Deborah Meier, 
which developed a graduation-by-portfolio system that became a model of authentic 
assessment. Several studies about CPESS and its sister elementary schools bred interest 
in and acceptance of new kinds of assessments. (Bensman,1994; Darling-Hammond, 
Ancess & Falk, 1995). 

With the formation of the Coalition of Essential Schools (CES) in 1984, authentic 
assessment became a more widely researched and disseminated idea. A significant 
number of the Coalition’s member schools began to use authentic assessments in their 
classrooms and, in some cases, for their graduation requirements. 

In the late 1980s and early 1990s there was a proliferation of articles in leading 
educational journals on authentic assessment, including notable contributions by Grant 
Wiggins, the former research director at CES, and researchers at the National Center for 
Restructuring Education, Schools and Teaching (NCREST) at Teachers College in New 
York. These articles promoted authentic assessment as an alternative to, and in some 
cases a replacement for, traditional testing. Professional development materials became 
widely available and the federal Department of Education made changes in the National 
Assessment of Educational Progress (NAEP) to reflect more authentic approaches 
(Haertel & Mullis, 1996). States began to include portfolios and other perfonnance-based 
components in their assessments, with Vermont and Kentucky going so far as to make 
portfolios the centerpiece of their graduation requirements. At the peak of the movement 
in the mid-90s, at least a dozen states were experimenting with some sort of authentic or 
performance-based assessment (Baron & Wolf, 1996). 

By the late 1990s the discussions around authentic assessment had shifted. While 
portfolios had become popular in individual classrooms, the use of authentic assessment 
on a larger scale was met with skepticism. Researchers found flaws in the reliability of 
scoring on the portfolio assessment systems of both Vermont and Kentucky, and there 
were concerns about inadequate training and excessive costs for both systems (Koretz, et. 
ah, 1994; Gong & Reidy, 1996). Research on authentic assessment shifted away from 
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advocacy and case studies to more quantitative examinations of its performance as a 
system. The personalized, open-ended, and contextual nature of authentic assessment 
appeared to be in conflict with the efficiency and reliability needs of large-scale 
assessment programs. With the coming of the curricular standards movement and high 
stakes testing, the federal government and states sought to standardize their assessment 
systems, mostly in favor of traditional forms of testing, leaving authentic assessment 
accepted as a pedagogical classroom tool but not as an assessment system (Koretz, 1998). 

Current Context of Authentic Assessments in the US and in Boston ’s Pilot Schools 

As traditional forms of assessment have regained popularity and assumed high stakes, 
authentic assessment has become somewhat symbolic of resistance to that trend. In New 
York, 35 schools have banded together as the New York Performance Standards 
Consortium to create a perfonnance-based alternative to the state’s traditional Regents 
exams (Davidson, 2002). In Massachusetts and other states, schools that have authentic 
assessments in place have had large percentages of parents and students boycott state 
exams and assert a need for multiple measures of student performance. They do so 
because they believe that the broad requirements of state content standards are 
antithetical to their belief in a more in-depth curriculum (a “less is more” culture, as Sizer 
might describe it). 

Some who sympathize with the goals of authentic assessment have concerns about the 
capacity of most schools to implement it effectively. Research has suggested that 
authentic assessment is educationally worthwhile but time consuming and vulnerable to 
misuse if adequate professional development is not provided (Stecher, 1998). Some of the 
most successful examples of these new assessments occur in either new schools or those 
with a sustained professional development and support program. Like most refonn ideas 
in education, successful authentic assessment programs seem to need support, time, and 
commitment in order to realize their promise. 

In Boston, authentic assessment has been a foundation of the Pilot Schools Network since 
its creation in 1994. The Network has developed a “Statement on Accountability” which 
guides the curriculum and assessment work of the schools (Appendix A). This statement 
outlines the Network’s views against standardization and promoting equitable 
opportunities for students. It also shares ten principles that guide teachers in their 
instruction, curriculum, and assessment practices. The Network schools have built shared 
culture based on authentic assessment and also exhibit success on many levels such as 
increased college acceptance rates, higher attendance, less violence, and better test scores 
compared to other Boston public schools (CCE, 2001). In the current political context, 
the debate over what constitutes sustainable quality assessment and the impact of testing 
on instruction has become extremely relevant to the life of every Pilot School. 
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RESEARCH QUESTIONS & METHODOLOGY 



This study focused on two major questions: 

• What does authentic assessment in the Boston Pilot Schools look like? 

• How has the implementation of authentic assessments strengthened the Pilot 
Schools? 

• What challenges do the Pilot Schools face as they further develop their use of 
authentic assessment? 

Data for this study were collected through observations, interviews, and document 
collection over two academic school years (2000-2002). Members of the research team 
visited nine of the then eleven Pilot Schools, serving as either observers or evaluators 
during authentic assessments of student work, for a total of twenty-six visits. Researchers 
documented the assessment a representative sample of students by race, gender, age, 
socioeconomic background, special education status, and academic achievement. Seven 
of the nine schools were observed three or more times, the others once apiece. Key 
documents were collected from each school, including rubrics, student handbooks, 
training guides, memos, student work, and explanations of the assessment system. 
Additionally, administrators and teachers at the highlighted schools were interviewed 
about each school’s assessment philosophy, process, accomplishments, and challenges. 

AUTHENTIC ASSESSMENTS IN THE BOSTON PILOT 
SCHOOLS NETWORK 

Authentic assessments are used in both ongoing curricula as well as culminating 
performances. In this study, we focus mostly on culminating authentic assessments in the 
Pilot Schools. While the formative assessment that happens daily in the classroom 
commands significant attention in Pilot Schools, we begin our documentation of Pilot 
School authentic assessment with the summative, concrete products observed in the 
culminating assessments at these schools. This portrait of culminating authentic 
assessments in the Pilot Schools Network covers school years 2000-01 and 2001-02. 
Below we articulate a series of observations about Pilot Schools’ use of authentic 
assessments. 
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Pilot Schools share common values in their development and use of authentic 
assessments 

While they have different formats and are used for different purposes, authentic 
assessments across the Pilot schools demonstrate many values in common. In the words 
of Newmann (1995), they exhibit construction of knowledge, disciplined inquiry, and 
value beyond school. Students present work from topics that allow for choice and self 
direction, so that the work reflects student interest and motivation. Students also 
demonstrate how they have acquired in-depth knowledge of the subject matter through 
formulating their own questions. 

Theme -based learning, in which a teacher offers multiple activities, honors the kids’ 
interests and where they are in their learning. Instead of trying to achieve a number [on a 
traditional test], students are figuring things out, finding answers to their questions [about 
the theme]. Portfolio coordinator 

One of the accountability principles in the Network is that assessment should “be 
embedded in curriculum and instruction that engages students in work that has a public 
purpose, that inspires students to become producers and contributors, and that assists 
them to become active participants in our democratic communities.” Teachers fashion 
their curriculum so that students reflect on the value of academic work to their learning 
and lives outside of school. 

Our thinking is to link the curriculum as much as we can with real life— so the projects 
and assessments. . . [are] something that a student might be expected to do in real life, so 
their performance is linked to being able to not only reflect on questions ahead of a 
presentation but be able to speak extemporaneously about something. 

Administrator 

Through rubrics and questions, teachers’ expectations should be made clear both to the 
students and to those evaluating the students’ work. In addition, all assessments involve 
the public and community beyond the school in some way. All of these shared values 
contribute to assessments that are authentic, that include reflection, the making of 
meaning, and relevance beyond school. 

Pilot Schools practice authentic assessment in multiple ways 

Our research revealed three major modes of authentic assessments, used differently 
depending on context and purpose - evaluations, presentations, and celebrations. All 
three of these modes of assessment can be called “authentic” because they are committed 
to asking students to engage in work that is “worthwhile, significant, and meaningful.” 
(Archbald & Newmann, 1988). 



Evaluations 

Evaluations are the most well known type of authentic assessment. All nine Pilot Schools 
studied use this mode of authentic assessment in some form. As practiced in Pilot 
Schools, evaluations are typically embedded in the curriculum and involve a student 
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presenting an aspect of academic work in order to exhibit mastery of content and skills. 
Most often, the work is compiled into a portfolio, or collection of the student’s best 
efforts over time and subject areas. During the presentation, the student speaks for an 
extended period of time, usually at a significant point in the academic calendar, to a panel 
of assessors, some of whom are drawn from beyond the school community. The assessors 
use a rubric to grade the student. The rubric clearly and publicly embodies the standards 
the student is held to, both in terms of presentation skills and in content knowledge. 

Often, important stakes are attached to student performance on evaluations, such as major 
grades, promotion, graduation, or honors. The student must reach a predetermined basic 
level of proficiency in order to advance. 

Comments from students support the effectiveness of the portfolio process in preparing 
them for college. This graduate of a Pilot high school describes how the high school 
evaluation process gave her practice for future college work. 

Academically, [the evaluation process] helped a lot, through portfolio reviews and 
presentations, because a lot of colleges ask you to do individual presentations, group 
presentations, or papers. So I had practice with all three [in high school]. I know [that 
when] a lot of students [at this college]. . .hear a professor ask them to do a 15 minute 
presentation, they panic. . .Thirty minute presentations are nothing to me because of [my 
Pilot School’s] high standards. I feel kind of overprepared. 

Pilot high school graduate 



Greater Egleston Community High School Graduation Portfolio Presentation. 

Graduating seniors choose three pieces of work to present from their portfolio, a 
collection of work spanning eight academic disciplines. For 40 minutes, they speak about 
their work and answer questions from a teacher, two outside evaluators such as 
community members or educators from other schools, and a recent Egleston graduate. 
Examples of work might be an autobiographical Powerpoint project from technology 
class, a portfolio of photographs taken and developed for a class, and a business plan for 
a restaurant from Economics. The panel spends 10 minutes scoring the presentation on 
their rubrics, and then discusses their marks with the student. Students meet the 
graduation requirements of the school by successfully passing the presentation, in 
combination with other work approved by classroom teachers. 



Mission Hill School Graduation Portfolio Review. To graduate, all 7th and 8th graders 
must present a collection of past and present work in six domains. They must also 
present and orally defend a finished piece of current work. Two teachers, an external 
evaluator, and a family member assess the work and the student’s competence in the 
presentation. For example, in a science review, a student presents a science experiment 
subjecting plants to cigarette smoke. A history review includes an essay and a rap on 
child labor (a topic chosen by the student). In every field students must show how to 
weigh conflicting evidence, take into account various viewpoints, make connections, and 
see how the work might matter to others. 
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Presentations 



Presentations are, in many ways, the precursors to evaluations. They are authentic 
assessments typically used in classes or school-wide throughout the year to assess 
ongoing performance rather than mastery and to provide students with feedback. All nine 
Pilot Schools studied use this mode of authentic assessment. They give a student 
experience in an authentic assessment environment while building a school culture of 
competent responses to authentic tasks. A student may present to his or her peers and be 
judged by a teacher or a small panel using a rubric. However, the presentation is less of a 
culminating event than a snapshot of continuing progress. 

1 have ways to gauge their daily understanding [of the novel they are reading]. For 
instance, I use storytelling. Rather than giving them a quiz, kids get up and [speak in the 
voice of one of the characters]. . .1 know that whatever they say, notice, and remember 
about the book is a reflection of what the theme is in their own life. I’m not preparing 
them just for college, I’m telling them it’s life skills, as well as skills that will serve them 
well in college or any learning situation. I build in the assessment pieces accordingly. In 
the culminating experience, I’m looking for pieces that force them to a level of self- 
reflection and insight to help them make some connections, because as we know, there 
are some adults who have never reflected on anything in their life. Teacher 



Fenway High School Math Exhibitions. In front of their peers, a teacher, and three 
outside evaluators from industry and the community, students present their solutions to 
open-ended math research questions. One student discusses the mathematical formula 
behind a card trick, another outlines the geometric progression of the rat population in 
Boston as it breeds, and a third student talks about the Cartesian coordinate system and its 
impact on the world. Students and adults ask questions and fdl out rubrics, which will 
ultimately affect the quarterly grades of all who present. 



New Mission High School Independent Learning Project (ILP) Presentation. A 

student stands before an audience of teachers, family members, fellow students, and 
administrators to explain a topic she chose to research: How do blind people cope with 
living in a seeing world? She queries the audience, challenging their assumptions about 
the blind, and offers alternative ways of serving the disabled from around the world. Her 
presentation encompasses an investigation of literature and science related to blindness. 
After fifty minutes, the audience fills out feedback fonns that will help guide the 
student’s future work at New Mission. 



Celebrations 

Celebrations are common at many schools. In Pilot Schools, the rigor and dedication 
necessary to successfully prepare for and complete them makes their performance a 
genuine form of authentic assessment. In fact, teachers noticing the high level of student 
engagement in dance recitals, science fairs, and sports events sought to replicate that kind 
of “authenticity” as a classroom experience, leading to the more structured modes of 
authentic assessment that are familiar today in these schools. While all nine Pilot schools 
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sponsor these kinds of events, at least three of the schools present them officially as 
authentic assessments, because they are public demonstrations of student work. 



Boston Arts Academy Student Choreography. Senior dance majors performed their 
own choreography based on research into an aspect of dance history; this perfonnance is 
an exit requirement for dance majors. Twelve pieces, mostly solos, were perfonned in the 
school’s Black Box Theater. Dancers and other arts professionals were present to ask 
questions and validate the work of the students, who had been cleared to present by their 
teachers. 



Boston Evening Academy End of Year Expo and Celebration. Students present 
autobiographical projects and meaningful academic work as a celebration of their 
accomplishments. Seniors discuss their future plans with outsiders. Teachers display 
interesting ideas from their classes and offer activities and projects. 



The following table summarizes the shared values and differentiating features of 
evaluations, presentations, and celebrations in the Pilot Schools. 



Shared Values: Authentic tasks, public presentation, personalized assessment, clear 
and visible expectations, community involvement, and student reflection. 


Evaluations 


Presentations 


Celebrations 


• Assess both content 


• Primarily used for 


• Have no stakes attached 


knowledge and 


giving feedback or lesser 


• Only participation is 


presentation skills 


grades 


assessed 


• Given at significant 


• Emphasize 


• Mark major moments in 


moments (end of unit, 


social/presentation skills 


a school year 


class or year) 


• Serve as early building 


• Serve as community 


• Have stakes attached 


blocks of authentic 


building event, not 


(grades, promotion, 


assessment culture 


academic assessment 


graduation, honors) 


• Assess student 


• Present student 


• Require mastery of 


performance of authentic 


responses to authentic 


authentic tasks 


tasks 


assignments 



Schools rely on authentic assessment to make serious decisions 

Of the nine schools studied, five use authentic assessments to help determine their 
students’ promotion or graduation. Three others use these assessments to provide 
feedback to students about the quality of their work or to assign honors or other 
distinctions. The final school is in the process of revamping its assessment structures. 
While other forms of traditional assessments are used in Pilot Schools, portfolios and 
other authentic assessments have become an equal or more dominant practice. Many 
schools count portfolios and exhibitions as significant portions of final course grades and 
most schools include authentic assessment results on report cards or transcripts. Overall, 
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authentic assessments are being used as a reliable and significant tool to set expectations 
and assess student achievement. 

It’s a real challenge for kids. It’s intense. And it feels like a huge accomplishment. So 
there’s this tone around getting work done that ideally, as we get better at it, is going to 
reduce in anxiety and increase in this sense of accomplishment and knowing what you’re 
doing and then having a great product. The public-ness of it contributes to the school 
culture, because people are presenting. . .this complicated work that has a lot of different 
facets, and people are trying to understand what [the students] understand. It makes the 
dialogue between kids and kids, and kids and teachers, much richer. If there isn’t the 
evaluation. . .if it’s not about assessment, some of that tension is not there, and you don’t 
get as far in that discussion. Administrator (K-8) 



Schools are at different stages in their development of authentic assessment systems 

Some schools have had more time than others to develop a comprehensive system of 
authentic assessment. One Pilot School is 20 years old, while several have opened since 
1997. In addition, some school directors have had more experience with authentic 
assessment than others. As a result, schools are at different stages in their implementation 
of authentic assessments, which range from emerging experiments to mature systems. 
Schools just starting to work with authentic assessment tend to use it for presentations in 
only a few subjects, have not adopted a school- wide system of performance-based 
requirements or evaluations, and have less developed rubrics and/or community 
involvement. Schools that are more experienced tend to add evaluations to their 
presentations, have developed school-wide systems of assessment with clear standards 
and stakes, and have oriented their curriculum around authentic assessment. They have 
moved from using authentic assessment as a small-scale pedagogical tool implemented 
by a few teachers to making it the cornerstone of a school-wide system. At least six 
schools increased their use of authentic assessment in the last year. Many have expanded 
the number of subjects or grade levels that use portfolios and exhibitions. 

A crucial element of authentic assessment systems is the rubric, which communicates to 
students what is expected and guides evaluators as they review work. Constructing good 
rubrics is difficult and takes time. At its core, a rubric embodies the values and mission of 
a school and reflects the clarity (or lack thereof) with which it pursues its purpose. A 
rubric telegraphs the priorities to students and judges. It conveys clear standards and 
expectations of a presentation to students and judges alike. It spells out what is expected 
in each category of presentation at every performance level. A good rubric leaves very 
little to interpretation. Quality rubrics can act as a scaffold for the entire class as they 
prepare for presentations. 

The rubrics we observed that were used in the Pilot Schools were at different stages of 
development. While we saw excellent examples of rubrics, different schools (and even 
departments or classes within schools) have constructed rubrics of varying quality. Some 
schools use standard rubrics for subject areas, while others develop rubrics for each 
project or topic. There is still work to do develop rubrics. 



Research and Evaluation Program 



11 



Center for Collaborative Education 




Authentic assessment can improve professional development and increase collegiality 

Several schools that have recently expanded their authentic assessment work indicated 
that it served as a catalyst for improved faculty collaboration and school-wide academic 
focus. Curriculum, rubrics, student support, and scheduling all had to be done 
collaboratively, resulting in the reorientation of faculty meetings, common planning time, 
and professional development. Schools with more mature assessment systems viewed 
such work as a centerpiece of their school’s identity, with authentic assessment 
penneating the school’s culture for both staff and students. 

People here have de -privatized their practice. [There is] a real awareness that the work is 
not easy and when you create it on your own, you don’t always hit the mark. 

They’re. . .in a trusting adult community enough to explore why and get some better 
answers and do it better the next time, so it’s helped in de-privatizing practice. That’s 
one key difference that authentic assessment has made to this community. It’s also 
pushed our desire for interdisciplinary curriculum. . .teachers are talking more with 
teachers of other disciplines so [teachers are] broadening their own perspectives and 
deepening their thinking. . .That comes out of the work of trying to arrive at authentic 
assessments. Administrator (High School) 

Authentic assessment engages the community in the school 

All nine Pilot Schools using authentic assessment invite members of the outside 
community to participate as judges at exhibitions, portfolio defenses, and other 
assessment events. These external judges have included professionals, government 
officials, parents, and community leaders. The purpose of having external judges is both 
to inform the public about the school’s work and to firmly ground assessments in the 
expectations of the community. For instance, at Fenway High School’s Science Fair, the 
audience included many scientists and academics whose daily practice was on topics 
students studied, in addition to adults from non-profit organizations and businesses who 
had high standards for presentations. 

At Greater Egleston Community High School’s portfolio presentations, the panels are 
made up of student alumni, community members, and professionals who all bring unique 
and high standards to the exercise. Students know this and respond with elevated levels 
of commitment and performance, a feat difficult to reproduce in a regular classroom 
setting. The presence of the “real world” in the school setting creates authenticity that 
students can both recognize and respond to. 

Summary 

We find that Pilot Schools have developed authentic assessments that address all three 
criteria developed by Newmann (1995): construction of knowledge, the use of 
disciplined inquiry, and value beyond school. Students construct knowledge in their 
presentations and portfolios, through teaching judges about early childhood education or 
researching paper topics in history or choreographing dance selections. They use 
disciplined inquiry when they discuss, elaborate, and defend their arguments and 
conclusions based on background knowledge and in depth understanding of concepts 
they have learned. The work that these assessments entail has value beyond school, as 
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evident in the involvement of community members in the reviews, and the community 
relevance of an environmental health project focused on the high incidence of asthma in 
the school’s neighborhood. Their varying stages of development and challenges in 
implementation attest to the complexity of this practice. 

Pilot Schools staff are not only able to embed authentic work through their curriculum, 
they are also able to successfully engage students such that they perform at high levels on 
a range of indicators (CCE, 2003). Authentic teaching and learning and high student 
performance go hand in hand in the Pilot schools. 



Research and Evaluation Program 



13 



Center for Collaborative Education 




CHALLENGES TO AUTHENTIC ASSESSMENTS IN THE 
PILOT SCHOOLS NETWORK 

Pilot Schools are committed to using authentic assessments as a way to understand what 
their students know and are able to do. They use such assessments to make important 
decisions about students’ promotion and graduation and to guide curriculum planning. 
Such comprehensive systems of assessment inevitably encounter challenges in 
implementation. We think that it is useful to share what we have learned about Pilot 
Schools’ implementation of authentic assessments for the following reasons — 1) to assist 
the Network schools in reflecting about their practices; and 2) to provide information 
about implementation which can be useful to practitioners, within the Network, within 
the district, and beyond. 

Not only are assessment systems difficult to implement, teachers and administrators in 
schools are usually working on improving many other aspects of practice simultaneously. 
Such work takes vision and commitment. School staff can sustain such a commitment 
when they are less isolated and know that others are also engaged in similar challenging 
work. They will also encounter fewer roadblocks if resources help them predict the turns 
the work will take. 

While Pilot Schools staff members engage in ongoing reflection about their teaching and 
learning within their schools and as a Network, sharing such challenges publicly takes 
courage. No one likes to have his/her practices criticized. But we should keep in mind 
that the original intent of Pilot Schools was to serve as laboratories of innovation for the 
district and beyond. It is also important to note that not all Pilot Schools have 
experienced the challenges to authentic assessments to the same degree. Below, we 
present our perspectives on the main challenges to authentic assessments 

Assessment of content knowledge needs to be improved 

One common element among the diversity of modes for authentic assessments shared 
above was their public-ness. All rubrics reviewed gave prominence and weight to the 
student’s skill and comfort in communicating his/her learning. One aspect of the 
authentic assessments that challenged some, but not all, schools was assessing the content 
knowledge of students. Teachers talked about the challenge of matching the assessment 
vehicle to the concept being taught: 

Content-wise there’s a tension around breadth vs. depth. Did [the teacher] design a great 
project? If so, is [the product] really evidence of what the kids have learned or is it more 
superficial? We try to design these projects and experiences, and what sometimes can 
happen is you create this really well-connected, involved, challenging bit of work that’s 
real, and because it has different elements, sometimes the concept is achieved but the 
content skills [such as statistics in math] don’t advance. . .It’s a real balance providing 
them the knowledge they need to understand the real world context, and still being able to 
spend enough time on the skill work. Administrator 

Teachers faced challenges in how to assess students whose work contains spelling or 
grammar errors or who fail to adequately respond to probing questions about content. 
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We found that rubrics often give more weight to presentation skills or process 
explanation than to demonstration of content knowledge or mastery. Even if evaluators 
have questions about content knowledge, the rubric might not give it much proportional 
weight, so that students with strong presentation skills might not be alerted to a potential 
area of improvement. Also, rubrics sometimes do not include clear criteria for assessing 
content knowledge, habits of mind, and understanding. 

Authentic assessments must balance the use of students ’ strengths and weaknesses 

Performance-based assessments allows for students to make use of their favored learning 
styles in a way that traditional assessments do not. One challenge that several 
interviewees cited was the balance between allowing students to choose favored modes of 
expression and pushing them to develop modes in which they are less developed. 

One of our goals is to help our students understand where their strengths and weaknesses 
are. Some of our students are very good orally, some of our students are not. Some of 
our students respond better when things are read to them, and some of them respond 
better when they can see them. So we work a lot on learning styles, so that all the 
authentic assessments that we do are based on helping students improve in areas they 
may be weaker in, but also to enhance their strengths. Administrator 

We observed a number of presentations where students gave excellent oral presentations 
that exceeded the quality of their writing. Panelists reviewing essays about the difference 
between lynching and terrorism or the political nature of the Olympics found that when 
students were asked about what they wrote, they were able to more clearly articulate their 
rationale and subtle nuances of meaning than was apparent in their writing. These 
observations underscore the Pilot principle of accountability that schools should “provide 
multiple ways of assessing student competency in meaningful ways, rather than relying 
on one single method.” For students whose verbal skills surpass their writing skills, the 
sharing of work through portfolio reviews or exhibitions may demonstrate to adults 
understandings not conveyed through writing. On the other hand, evaluations of student 
writing should meet the school’s expectations and standards in that area. 

Not all performance-based assessments in Pilot Schools are authentic assessments 

As Meier (1993) notes, not all performance-based tasks are authentic. Students at some of 
our observations presented work that was drawn from textbooks or MCAS-preparation 
exercises, in addition to presenting work that lacked a real world or authentic context. 
Examples included science fair projects based on or adapted from tasks described on web 
sites or in science workbooks. Asking students to construct knowledge and demonstrate 
their understanding is important and valuable. However, bringing students as far into the 
complexity of real life as possible is essential if students are to be highly engaged and 
authentic assessment is to be truly effective. Schools should aspire to as much 
authenticity as possible when constructing both their assignments and their assessments. 
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Outside evaluators need more support 

When Pilot Schools open their assessments to judges from the community, they add a 
crucial element of authenticity to their work by exposing students to the standards of the 
outside world. Because Pilot Schools want to prepare their graduates to thrive beyond the 
school walls, the community plays a vital role in determining the readiness of graduates 
to meet the standards of both the school and of life beyond school. The commitment of 
all nine schools we observed to these ideals was visible in all participants, who were very 
aware of the positive presence of outsiders in the assessment process, lending weight and 
importance to the event for all involved. 

Because outside evaluators are so important to the authentic assessment process, they 
need to be provided as much training and support as possible in order to best perform 
their duties. Most schools sent letters to outsiders that included information about the 
school and often a copy of the rubric that would be used by the panel. Some schools 
offered meals and receptions to make guests feel welcome. These schools also sent home 
surveys for feedback or thank you notes to their guests. Most importantly, several schools 
had training sessions for evaluators to review rubrics, rules, and the background of the 
school. 

Outside evaluators with the most complete infonnation about the process and schools’ 
expectations of them are better able to assess students. Schools must assume that external 
evaluators are not familiar with their processes and contexts, especially when assessments 
are revised yearly. The more energy that is put into educating and supporting outside 
evaluators, the more power, benefit, and authenticity the assessment process will have. 
The use of outside evaluators should be accompanied by the following considerations: 

• Outside evaluators, or even school staff who teach different subjects, are not always 
knowledgeable in the areas they are judging. In atypical subjects, like technology or 
the arts, lack of content knowledge among evaluators results in less probing questions 
and acceptance of student responses without substantial fact checking. 

• At some schools, there are either too many subjects being presented at one time or not 
enough time given to adequately investigate the quality of a piece. Evaluators 
unfamiliar with the school and seeing several students at once may find it difficult to 
accurately or fairly judge the content being presented. 

• When some evaluators know the student and others do not it can create an imbalance 
on the panel that can affect assessment. We observed a number of situations in which 
a panel would defer to a teacher who knew the student, especially when the panel had 
concerns about a student’s level of content knowledge. The differing familiarity with 
the student among examiners should not compromise the important objectivity that 
those from the outside bring to this process, which is central to the “authenticity” of 
the assessment. 
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Rubrics, for the most part, gave clear and helpful guidance on how an individual should 
score a presentation. All schools should spell out their grading procedures for authentic 
assessments in as much detail as possible. Such clarity and consistency will optimize the 
involvement of community members in the process. However, at several schools, staff 
told guest evaluators that their scores would be “taken into account,” but the weight of 
their scores was unclear. In some instances, a policy for combining the marks of school 
staff, external evaluators, or, in some cases, students was not in place. Some schools 
explicitly used group consensus as their model, allowing time for groups to discuss and 
arrive at a shared understanding about a student’s work. However, some schools had 
allotted little time for that kind of in-depth discussion and had no formal guidelines for 
what to do when discrepancies in grading occurred. 

Concerns about reliability and standardization of scoring need to be addressed 

Researchers have expressed concern about the reliability of authentic assessments 
(Koretz, 1998; Stecher, 1998), and this issue is also important at the school level. Schools 
could collect data that could illuminate trends or issues in regards to scores. This data 
could include a comparison of student performance on authentic assessments with more 
traditional teacher-designed test scores or with standardized test scores. Additionally, 
schools could keep track of how scores of authentic assessments vary by year or subject 
or teacher, as well as by gender, age, or race, to provide useful information for targeting 
the improvement efforts. 

IMPLICATIONS FOR SCHOOLS IMPLEMENTING 
AUTHENTIC ASSESSMENTS 

Pilot Schools are committed to developing authentic assessments to collect evidence as to 
how their students meet the expectations of teachers, families, and the community at 
large. Because they are small, vision-driven, and autonomous, these schools may 
develop curriculum and assessments with their own students and standards in mind, 
rather than responding to district curriculum mandates or professional development 
initiatives. As these schools believe in “less is more” and “depth over breadth” in 
teaching and learning, authentic assessments meet their needs more readily than high- 
stakes, standardized tests. 

Our findings have positive implications for all stakeholders: 

• Authentic assessments raise the stakes of learning for students. Students in Pilot 
Schools have opportunities to present publicly to adults and students, both familiar 
and not, throughout their school careers. They complete work that is relevant to their 
daily lives and contexts. They can demonstrate what they know and can do in 
multiple ways, rather than just through a paper and pencil test. 

Authentic assessment gives me a better idea of my kids, because everyone has to meet the 
same requirement. In a regular school, that does not always happen. In a regular school, 
if there’s a science fair, maybe everyone doesn’t have to present. In a regular school, if 1 
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give a test and 10% fail, they just get a failing grade. If a child fails a portfolio, they have 
to go back and redo it. It is a more equitable system. Teacher 



• Authentic assessments improve curriculum and instruction. Teachers have 
opportunities to refine their curriculum and assessments collaboratively, often across 
subjects and grade levels. They de-privatize their practice by sharing their 
assignments and assessments publicly. 

Authentic assessments affect teacher planning. When we talk about authentic 
assessments and exhibitions, it’s really important to be clear on what teachers want 
students to do, and that’s affected our time together because those conversations have to 
take place. . .For example, our teams work together. What is it that we want students to 
go out into the world with? What is the mission statement for the department? What are 
the essential skills we want them to have? It’s played a major role in curriculum and 
experiences for students. . .It’s planning backwards. Administrator (High School) 

• Authentic assessments make public the work of schools. Community members 
become more educated about public schools, curriculum, and assessment. They lend 
credibility to the authentic assessments and lend them a sense of seriousness and 
purpose. 

The Pilot Schools Network is strengthened as member schools share their progress on 
authentic assessments through school visits, serving on each others’ review panels, 
conference presentations, and leadership meetings. In this way, authentic assessments 
may provide the foundation for a system of assessments as a viable alternative to 
standardized testing. The Network allows for increased communication among schools, 
and this increased communication strengthens the knowledge and commitment within the 
Network to designing and implementing solid authentic assessment systems. 

The Boston Public Schools and other districts may leam from the Pilot Schools’ use of 
curriculum and assessment autonomy. These schools serve as models for other schools 
and districts interested in developing authentic assessments. Shared knowledge can 
reduce the need for schools to reinvent the details of their assessment systems. Schools 
can pursue such sharing through visits to other buildings, staff participation in judging at 
other sites, and the exchange of rubrics, examples of student work, and documentation of 
the assessment, promotion, and graduation requirements at each school. 

Future studies by CCE will describe the more formative authentic assessments used in 
Pilot Schools, the impact of authentic assessments on instructional practice and 
curriculum development, and the perspectives of teachers and students on authentic 
assessments as practiced in Pilot Schools. 
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APPENDIX A: PILOT SCHOOLS NETWORK 
STATEMENT ON ACCOUNTABILITY 

The Pilot Schools believe that having in place a strong system of assessing student 
progress is vital to creating excellent schools in which all students learn and achieve at 
high levels. We believe in standards that lead to excellent schools, not standardization. 
We support the development of network-wide competencies and assessments that, while 
providing common information on how schools are doing, also allow for and encourage 
uniqueness in approaches to instruction and assessment among schools. Ultimately, good 
assessment systems should open doors for all students rather than shut them, and help 
students graduate with a range of options. The Pilot Schools Network assessment system 
is built upon the following principles. Assessment should encompass the following: 

• Provide multiple ways of assessing student competency in meaningful ways, 
rather than relying on one single method 

• Eliminate secrecy, so that all students, families, and the public understand the 
knowledge, skills, and habits of mind that students are expected to know and be 
able to do, how they will be expected to demonstrate this knowledge, and what 
constitutes high quality work 

• Be developed and used by those working most closely with students, while also 
involving families and the community 

• Provide information to students, families, and the community on how students are 
progressing toward meeting goals 

• Be embedded in curriculum and instruction that engages students in work that has 
a public purpose, that inspires students to become producers and contributors, and 
that assists them to become active participants in our democratic communities 

• Help students become independent, self-reliant, and thoughtful learners, and gain 
a sense that they are able to affect and improve the world around them 

• Provide opportunities for students to be successful, to leam from mistakes and 
challenges, and to build persistence and resiliency as learners 

• Help students become reflective learners and self-assessors who monitor their 
own growth, build on their strengths, and develop their skills 

• Promote reflective practice in teachers, leading to improved instruction 

• Reflect the best research on instruction and assessment 
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